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ABSTRACT 

Surveys searching for transiting exoplanets have found many more candidates than they have been 
able to confirm as true planets. This situation is especially acute with the Kepler survey, which 
has found over 2300 candidates but has confirmed only 77 planets to date. I present here a general 
procedure that can quickly be applied to any planet candidate to calculate its false positive probability. 
This procedure takes into account the period, depth, duration, and shape of the signal; the colors of 
the target star; arbitrary spectroscopic or imaging follow-up observations; and informed assumptions 
about the populations and distributions of field stars and multiple-star properties. Applying these 
methods to a sample of known Kepler planets, I demonstrate that many signals can be validated 
with very limited follow-up observations: in most cases with only a spectrum and an AO image. 
Additionally, I demonstrate that this procedure can reliably identify false positive signals. Because of 
the computational efficiency of this analysis, it is feasible to apply it to all Kepler planet candidates 
in the near future, and it will streamline the follow-up efforts for Kepler and other current and future 
transit surveys. 



1. INTRODUCTION 

The first exoplanets found to transit the face of their 
host stars were all initially discovered by Doppler sur- 
veys, which detected minute radial velocity (RV) vari- 
ations of stars indicative of the gravitational tug of or- 
biting planets (Charbonneau et al. 2000 Henry et al. 



2000 



2005) 



|Butler et al.pti04[ |Bouchy et al.||2a05| |Sato et ar 

Surveys design ed to detect ex oplanets via their 



transits soon followed (|Alonso et al.||2004[ |McCul lough 
et al |[2006{ |Bakos et al.||20U7| |Collier Cameron et al. 
2007 ) , motivated by the wealth of physical information 



that the light curve of a transi ting planet can provide 
( |Seager fc Mallen-Ornelas||2003[ ). While initial forecasts 
suggested that such programs would churn out planet 
detections at tremendous rates ( Horne|[2003 ) , the actual 
yield of ground-based surveys has been much more mod- 
est, with only about a dozen discoveries published in the 
first five years of transit survey operations. 

One of the major reasons for this slower-than-hoped- 
for discovery rate (in add ition to the various con tribut- 
ing subtleties discussed in |Beatty fc Gaudi] ( |2008[ )) is the 
preponderance of various eclipsing stellar binary scenar- 
ios mimicking planet transit signals, or so-called "astro- 
physical false positives," combined with the quantity and 
expense of the observational resources required to con- 
firm a signal as truly planetary. The classic example 
of an a strophysical false po sitive is the case of OGLE- 
TR-33 ( |Torres et aT] |2004| ) . Identified by the Optical 
Gravitational Lensing Experiment (OGLE) as a transit- 
ing planet candidate with a periodic photometric dip of 
2%, subsequent follow-up observations seemed only to 
confirm its planetary nature: periodic RV variations were 
discovered in phase with the transit signal, no nearby 
stars were observed to be blended within the photomet- 
ric aperture, and even the eclipse depth was measured 
to be color-independent. However, further investigation 
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revealed that various blended eclipsing binary scenarios 
could fit the light curve just as well as a transiting planet 
model, and upon careful inspection of the spectral time- 
series data it became clear that OGLE-TR-33 was in- 
deed a hierarchical triple system containing an eclipsing 
binary, and not a transiting planet at all — the appar- 
ent Doppler shift of the lines was actually an asymmetry 
in the combined line profiles of two stars. There were 
early isolated attempts to statistically quanti fy the prob- 
abilit y that signals might be false positives (Sahu et al 
2006 e.g.), but this and other similar exam ples, in addi- 
tion to theoretical predictions ( Brown|2003 1 , were strong 
early warnings to the community that false positive sce- 
narios may lurk behind any transit candidate, and that 
extremely careful analysis is required to rule out false 
positives in any individual case. 

As a result, the traditional method of promoting a 
transit candidate signal to the status of a bona fide planet 
involves an in-depth, multi-instrument follow-up cam- 
paign. A prototypical example of this pr ocedure is de- 
tailed in the discovery paper of TrES-lb (Alonso et al 
2004 1 , which was the first transiting exoplanet to be dis- 
covered by a survey specifically designed for the purpose. 
The follow-up observations required to confirm this sig- 
nal as a planet were the following: 

• H- and X-band adaptive optics (AO) imaging to 
rule out contaminating stars of similar brightness 
outside of ^0'.'3 separation from the target star. 

• Medium-resolution Echelle spectroscopy (velocity 
resolution of 8.5 km s -1 ) at seven epochs, to rule 
out scenarios involving massive binary companions 
and to characterize the host star as suitable for 
further follow-up. 

• Multi-color transit photometry with three different 
facilities in seven different filters (in addition to the 
discovery data), to check that the eclipse depth is 
color- independent, consistent with the eclipse of a 
cold, substellar object. 
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• High-precision (~10 m s" 1 ) RV measurements with 
the Keck High Resolution Echelle Spectrometer 
(HIRES) at eight epochs, which detected the sinu- 
soidal Doppler variation caused by the transiting 
planet. 

The "funnel" procedure established with this discovery, 
in which a large number of candidatesjare whittled down 
by various follow-up observations to identify the few that 
are worthy of observations with a large telescope such 
as Keck, was absolutely necessary in the early days of 
transit surveys, given the scarcity of precise RV spec- 
trometers and the desire to discover and characterize 
individual exoplanetary systems. Throughout the last 
decade this has been widely adopted as the follow-up 
paradigm: find many candidates, identify a subset ap- 
propriate for follow-up, and eventually confirm a small 
number of these with RV measurements. Understand- 
ably, most of the exoplanets detected in this manner 
have been hot Jupiters, given that both their photometric 
(detection) and Doppler (confirmation) signals are more 
readily measurable tha n those of smaller pla nets. 

The Kepler mission ( Borucki et al. 2008 ) poses com- 
pletely different problems, both in scale and specifics, 
than do ground-based surveys. Not only is the sheer vol- 
ume of candidates co mpletely unprecedented (over 2300 
( Batalha et al.|[2012[ )), but the keystone of traditional 
confirmation (RV measurements) is literally impossible 
to obtain for most of the candidates, given current and 
immediately foreseeable telescope resources: the stars are 
too faint and/or the planets are too small. Systems that 
show multiple transiting candidates sometimes display 
transit timing variations (TTVs) that may be used to 
dynamically confirm the planets in the system ( Holman 
et aL|2010||Lissauer et al.|2011||Cochran et al.|2011t|Fab- 
rycky et al.||2012| [Ford et al.||2012| Ptcffcn ct al. 2012), 
but the number of systems for which this is feasible is 
still an overwhelming minority of all the candidates. 

Given these difficulties, it has become necessary to 
adopt a new paradigm of transiting planet confirma- 
tion: probabilistic validation. The philosophy behind this 
strategy is to demonstrate that a particular transit can- 
didate is much more likely to be a transiting planet than 
it is to be a false positive — which may be the only option 
available if positive confirmation via RVs or TTVs is im- 
possible or impractical. This has been exemplified by the 
Kepler team's BLENDER procedure, which is descended 
from the methods which helped identify OGLE-TR-33 
as a false positive, and also used to help confirm other 
OGLE candidates (Torres et al. 2005| e.g.). Using a suite 
of follow-up spectroscopic and imaging observations com- 
bined with extensive light curve fitting, BLENDER rules 
out regions of parameter space where false positive mod- 
els are unable to fit the photometric data as well as a 
planet mo del. First demonstra ted with the validation of 
Kepler-9d (Torres et al. 2011), this procedure has been 
used to validate many of the planets which have so far 
received official Kepler number designations. 

However, despite the successes of BLENDER, Kepler 
still has a problem of scale: BLENDER is expensive, 
both in CPU- and person-hours, and has not yet been ap- 

2 Tres-lb was one of 16 transit-like signals among 12,000 stars 
monitored by the survey in its field and the only one to become 
positively identified as a planet 



plied to large numbers of candidates in a wholesale man- 
ner. As the overall mission of the Kepler project is not 
individual planet detections (as has been the case with 
ground-based surveys) but population statistics, there 
was a need to estimate the a priori false positive probabil- 
ities (FPPs) for Kepler candidates, before any follow-up 
observation or detailed BLENDER- style analysis. This 
was th e inspiration for the work of Morton & Johnson 
(20111, which demonstrated that for transit candidates 
that have passed the vetting tests that are possible with 
Kepler photometry alone, the FPP is typically around 
5% or lessj^j For broad-brush studies to date, this result 
has been used to essentially ignore false positive contam- 
ination when analyzing the statistical properties of the 
candidate sample. 

Despite the reassurance these a priori calculations pro- 
vide that the broad characteristics of the overall Kepler 
candidate sample reflect those of the true exoplanet pop- 
ulation, there remains significant motivation to push a 
priori FPP analysis beyond MJ11. First of all, MJ11 
did not fully quantify all false positive scenarios; in par- 
ticular, grazing eclipse signals, non-blended eclipsing bi- 
naries, and background blended transiting planets were 
not considered. A more thorough analysis will further 
increase the fidelity of the entire Kepler sample and thus 
enable more nuanced statistical analyses. 

Secondly, the framework presented in MJ11 uses only 
minimal information about the photometric signals; that 
is, their depth and period. Much more information is 
available, particularly regarding the shape of the tran- 
sit. In addition, MJ11 did not provide a transparent way 
to incorporate the results of follow-up observations to 
update the a priori calculations. And finally, as the pro- 
cess of probabilistically validating a planet is the same 
as demonstrating that its FPP is sufficiently low, a more 
stringent FPP calculation in the style of MJ11 may be 
used to validate large numbers of transit candidates for 
which traditional confirmation is difficult or impossible. 
In addition to enabling ground-based surveys to stream- 
line follow-up efforts, such a tool could ensure that the 
legacy of the Kepler mission is indeed thousands of tran- 
siting planets, not just planet candidates. 

This paper presents a comprehensive transit candidate 
validation procedure that is based on the a priori frame- 
work of MJ11, yet also uses information about the shape 
of the transit signal and naturally incorporates (yet does 
not require) follow-up spectroscopic and imaging obser- 
vations. An early version of this analysis, representing 
a halfway point between MJ11 a nd this work, was used 
to validate the KOI-961 planets ( |Muirhead et al.||2012 ). 
Crucially, this method is capable of analyzing large num- 
bers of candidates quickly; the entire end-to-end com- 

3 The dominant reason why this is much lower than has been 
the case for ground-based surveys is because the quality of the Ke- 
pler photometry allows for much more constraining prc-followup 
vetting than has been possible with ground-based surveys; in par- 
ticular, Kepler is able to identify many blended binary scenarios 
by measuring the target's center-of-light to shift during eclipse. 
This makes the effective "blend aperture" for Kepler much smaller 
than for ground-based wide-field surveys. Additionally, the pre- 
cise photometry can put strict limits on the presence of any sec- 
ondary eclipse, which further constrains false positive scenarios. 
And finally, the signals are typically shallower than detected in 
ground-based surveys, which also contributes to intrinsically lower 
FP rates. 
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putation for a single transit candidate signal takes only 
about 10 minutes on a typical personal computer. It is 
thus capable of being applied to the entire Kepler candi- 
date sample in the immediately foreseeable future. Look- 
ing beyond Kepler, the larger goal of this procedure is to 
revolutionize the follow-up strategy of large-scale transit 
surveys by reducing the time and and cost necessary to 
identify false positives or confirm transiting planets, and 
to enable statistical analyses of transit candidate popu- 
lations without having to wait for every candidate to be 
individually solved. 

In Sj2]l review the probabilistic framework introduced 
in MJ11 and summarize the false positive scenarios con- 
sidered in this work. I describe the entire procedure in 
detail in Sj3j present the results of applying this proce- 
dure to a number of previously studied Kepler signals in 
[j4j discuss the relationship of this work to MJ11 in f|5j 
and provide concluding remarks in fj6] 

2. FRAMEWORK 

Validating a transiting planet means demonstrating 
that the false positive probability (FPP) of the signal is 
small enough to be considered negligible. Regardless of 
exactly where this threshold lies, the process of validation 
is the same as carefully calculating the FPP. Following 
MJ11, I define the FPP for a given transit signal: 

FPP = 1 - Pr (planet | signal) (1) 

where the vertical line means 'given' (i.e. "the probability 
of there being a planet given the observed signal"), and 

Pr(planet | signal) = — . (2) 

In the above equation, C represents the Bayesian like- 
lihood factors (the "probability of the data given the 
model"), and 7r represents the priors (the a priori prob- 
abilities that each given scenario exists). The TP sub- 
script represents the true transiting planet scenario, or 
"true positive," and FP represents all of the false positive 
scenarios. 

The false positive scenarios I consider in this analysis 
are the following: 

• A non-associated foreground or background eclips- 
ing binary system is blended within the photomet- 
ric aperture of the target star (BEB). 

• The target is a hierarchical triple system in which 
two of the components eclipse (HEB). 

• The target star is an eclipsing binary (EB). 

• A non-associated blended foreground or back- 
ground star happens to have a transiting planet 
(Bpl). 

This list is more comprehensive than MJ11, which only 
considered BEBs and HEBs quantitatively, and com- 
prises every conceivable astrophysical false positive sce- 
nario. 

3. PROCEDURE 

The validation procedure I present in this paper has 
the following four steps: 



1. Simulate a representative population for each 
scenario, fixing the period to be that of the transit 
candidate signal. 

2. Use this population to calculate the prior for each 
scenario, optionally taking into account any follow- 
up observations that may exist. 

3. Use the same simulated population to calculate the 
likelihood of the observed transit signal under each 
scenario. 

4. Combine these numbers to calculate the FPP (Eq. 
[lj of the signal under an assumption of the specific 
planet occurrence rate; if this number is signif- 
icantly <1% for a conservative estimate of planet 
occurrence, then consider the planet validated. 

In the following subsections I explain each of these steps 
in detail, including the important terms in boldface 
above. 

3.1. Population Simulations 

Central to the validation procedure I present in this 
paper is the idea of simulating "representative popula- 
tions." This means using physically and observationally 
informed assumptions to generate a population of differ- 
ent instances of a particular transit or eclipse scenario, 
at a given fixed period, such that the simulated popu- 
lation accurately reflects the true population. In other 
words, a representative population must be created such 
that each instance in the population is equally likely to 
exist; this makes averaging over the population trivial, 
as each instance has equal weight in defining the dis- 
tribution of any desired quantity. For clarification, in 
this paper the following boldfaced words conform to this 
usage: a representative population for a particular 
scenario (e.g. BEB) is made up of many simulated in- 
stances of that scenario. For all the results presented 
in this paper, I create each representative population to 
have 20,000 instances. 

These simulations necessarily require several assump- 
tions: 

• The population of stars in a cone around the 
line of sight. This requires assumptions about 
both Galactic structure and stellar evolution. TRI- 
LEGAL (TRIdimensional modeL of thE GALaxy; 
Girardi et al. 2005, and used in MJ11) is a tool per- 
fectly suited to this purpose. TRILEGAL takes as 
input a given direction and area on the sky, a value 
for extinction at infinity, and parameterizations of 
the Galactic structure (e.g. scale height of the disk), 
and returns an appropriately simulated stellar pop- 
ulation, with apparent magnitudes in any chosen 
photometric system as well as the physical proper- 
ties of all the stars. 

• The properties of multiple star systems. This 
encompasses both multiplicity fractions and the 
distributions of physical and orbital properties for 
multiple systems, such as the mass ratio and ec- 
centricity distributions. All these properties may 
be adopted based on the statis tics of observa- 
tional multiplicity surveys, such as[Raghavan et al. 
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(20101. To simulate the physical distributions of 
multiple systems, I first start with a total mass of 
a star system, split it into two components with 
50% probability, and then split one of those into 
two more with 25% probability. Each time I make 
a split I choose a mass ratio uniformly between 
0.1 and 1. There is no specific physical motivation 
for this proc edure, but it reproduc es well the ob- 



servations of Raghavan et al. 



(2010), who observed 



roughly 35-40% of stars to be in binary systems and 
about 12% to be in triple or higher-order multiple 
systems, and a flat mass ratio distribution for bi- 
nary pairs. Besides the binary fraction, I also must 
assume a "close binary fraction," which is the frac- 
tion of stars that have companions with periods 
small enough to mimic the observed planet tran- 
sit candidate population. For this cutoff I choose 
P < 300 day s; according to the perio d distribution 
observed by Raghavan et al.| (2010) this encom- 
passes about 12% of binary stars. Therefore I as- 
sume a "close binary fraction" of 0.05 rs 0.4 x 0.12. 
Unlike the binary fraction, we do not use a pared- 
down "close hierarchical fraction," under the as- 
sumption that most systems in triple and higher- 
order systems will undergo significant orbital evo- 
lution over their lifetimes, ending up with at least 
one close pair — and according to stu dies that have 
observed this (Tokovinin et al. 2006). 



• Stellar models. Accurately simulating eclipsing 
star systems requires assigning appropriate phys- 
ical and observational properties to simulated bi- 
nary companions. Given a primary star of a par- 
ticular mass, age, and metallicity, a simulated bi- 
nary companion is assigned a mass according to 
the assumed mass ratio distribution, and then a 
radius and magnitudes in different passbands ac- 
cording to stellar model predictions for that par- 
ticular mass, age, and metallicity. Models are also 
required to accurately simulate a target star as a 
hierarchical system, which requires estimating ab- 
solute magnitudes for the target star in addition to 
its companions, in order to calculate both blended 
eclipse depths and the total colors of the syst em. 
In this work I use the Padova stellar models (Gi- 
rardi et alT||2002[ ) for all masses >0.15 M (tne 
lower limit returned by the online interface that 
provides the model s) and the Dartmouth models 
( potter et aT||2008| ) below 0.15 M . Preference is 
given to the Padova models for the sake of consis- 
tency, because they are used to populate the TRI- 
LEGAL simulations. Below 0.11 Mq (the lower 
li mit for the Dartmout h models), I use the models 



of [Baraffe et al.| p002| 



• Planet occurrence rate. Somewhat paradoxi- 
cally, in order to complete the FPP calculation for 
any given transit signal, prior assumptions about 
the planet occurrence rate and radius distribution 
must be made. See section 3.4 for how I treat this 
delicate issue by introducing the concept of the 
specific occurrence rate. For the background 
transiting planet scenario, I use a generic assump- 
tion of a 40% planet occurrence rate, with a power 



law distribution of planet radii dN/dR oc R 2 . 

Due to all these assumptions, the procedure I present 
in this paper is model-dependent. However, this is a fea- 
ture of all Bayesian analysis, which always depends on as- 
sumptions (priors) and models, whether physical or em- 
pirical. The advantage of the Bayesian approach is that 
it provides a natural way to incorporate the existing body 
of knowledge into a new analysis, and to make assump- 
tions explicit. That is, there has been significant work in 
astronomy dedicated to investigating Galactic structure, 
stellar models, and stellar multiplicity, and the strategy 
of this analysis is to attack this probabilistic validation 
problem armed with as much of this prior knowledge as 
possible. 

3.2. Priors 

I define the prior (7^) for each scenario to be the prod- 
uct of three distinct factors: the probability that the as- 
trophysical scenario in question exists within the photo- 
metric aperture, the geometric probability that the orbit 
is aligned such that an eclipse is visible, and the proba- 
bility that the eclipse is "appropriate" — that is, able to 
mimic (or be detected as) a transiting planet. I calculate 
the first of these factors based on the Galactic popula- 
tio n an d stellar multiplicity rate assumptions discussed 
in £3.1 combined with knowledge of the sky area inside 
which a FP might reside. 

The probability of an eclipse occurring, given random 
orbital orientations with respect to the plane of the sky, 

pr _ d _ ^ { ^ ) (3) 

where R\ and R2 are the radii of the two bodies in ques- 
tion; a, e and w are their orbital semimajor axis, eccen- 
tricity, and argument of periastron; and the ± symbol is a 
+ for primary eclipse and — for secondary)^] Calculating 
this probability for a particular scenario requires aver- 
aging over the distribution of the various physical and 
orbital quantities of that particular scenario (i.e. larger 
stars are disproportionately likely to host a binary star 
that eclipses, for a given period); in practice this is ac- 
complished simply by simulating a representative popu- 
lation (§3TJ) for a particular scenario, assigning isotropic 
orbital inclinations, and counting what fraction result in 
an eclipse (either primary, secondary, or both). 

In order for an eclipse of a certain scenario to be "ap- 
propriate," it must pass all vetting constraints available, 
and the probability that a certain scenario will result in 
an appropriate eclipse may be calculated by just counting 
what fraction of a representative population pass the con- 
straints. For the analysis in MJ11, the only constraint 
we used was depth: the diluted primary eclipse depth 
must be deep enough to be detected, while the secondary 
eclipse depth must be shallow enough not to be detected, 
given a detection threshold (or one of the eclipses must be 

4 Note that using the sum of the radii in this equation for transit 
probability accounts for the possibility of grazing eclipses; this is 
different from MJ11, who used the difference of the radii, explicitly 
rejecting grazing eclipses as potential false positives, on the grounds 
that any explicitly V-shaped transit could be vetted as a probable 
false positive based on photometry alone. In this work, I include 
the possibility of grazing transits in all the scenarios. 
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missing due to an eccentric orbit). In the present work, 
I have generalized the framework in order to naturally 
include different types of supplementary observations — 
either preparatory or follow-up, as the case may be. 

For example, as part of the simulations, I assign each 
false positive instance apparent magnitudes in many 
passbands. This allows observed colors of the target star 
to be used to constrain the population simulations. For 
example, in the case of Kepler targets, each star has 
Kepler Input Catalog (KIC) griz + 2MASS JHK mag- 
nitudes, and we constrain our representative populations 
of HEB and EB scenarios to conform to the observed to- 
tal g — r and J — K colors, to within 0.1 mag (I choose 
these two colors in particular, and not g — K, for ex- 
ample, to avoid being t oo sensitive to metallicity effects 
(Johnson & Apps 2009)). This results in asymmetric dis- 
tributions ot target star properties, as shown in Figure [T] 
If a high-resolution spectrum is obtained for the target 
star, allowing for spectroscopic measurements of T e ff and 
log g of the star that dominates the system luminosity, 
then these populations may be constrained to require the 
primary star to have the measured properties. 





Kepler-19 
L-j Hierarchcial triple scenarios 
allowed by KIC colors 
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Fig. 1. — The distribution of hierarchical triple systems allowed 
for Kepler-19b (KOI 84.01) by the KIC g-r and 2MASS J-K col- 
ors. Note how there is a tail of higher-mass systems that can mimic 
Solar-like colors. This is important for calculating FPP, because 
systems with higher-mass (and thus higher-luminosity) primaries 
are more prone to cause false positives with shapes similar to planet 
transits. If the spectroscopic properties of the primary are mea- 
sured (e.g. with a high-resolution spectrum), then this significantly 
constrains the false positive landscape (see Figure [6|. 

In addition, for each blended false positive instance 
I assign an angular separation from the target star on 
the sky, assigned differently for different scenarios. An 
important quantity for this step is the so-called "con- 
fusion radius," or the separation from the target star 
inside which a blend might reside. This may be con- 
strained either by centroid analysis or simply by the size 
of the aperture, to be most conservative. Chance-aligned 
blends are assigned positions perfectly uniformly within 
the confusion radius surrounding the target star, whereas 
the positions of hierarchical scenarios are assigned ac- 
cording to the assumed orbital distributions and random 
mean anomalies. 

Using these separations in combination with the multi- 



band photometry information, I may use a contrast curve 
(magnitude contrast as a function of radius from the tar- 
get star) that results from analyzing a high-resolution 
follow-up image in any passband to rule out all instances 
of a particular scenario that could have been detected by 
the observation. The same principle may be naturally 
used to rule out instances based on other types of follow- 
up, such as multi-color transit observations testing for 
color-dependent transit depth. After including all avail- 
able supplementary observational information — or none, 
as the case may be, in which case the only constraints 
are the primary/secondary depth conditions and the con- 
fusion radius — the "appropriate probability" is then the 
fraction of instances of a particular scenario that pass all 
the vetting tests. 

3.3. Likelihoods 

Once the representative population for a given scenario 
is simulated and vetted according to whatever informa- 
tion is available, then the likelihood C for a given transit 
signal under that scenario may be calculated. It is at 
this stage in the procedure that the method introduced 
in this paper advances most significantly beyond the pre- 
liminary work of MJ11, so I will first give a brief review 
of how we treated likelihoods in that work. 

In a Bayesian calculation, the likelihood factor may 
be colloquially described as "the probability of the data 
given the model." In the simplest case, if the data con- 
sists of a single measurement, and the property being 
measured follows a known probability distribution, then 
the likelihood of the data is simply the known probabil- 
ity density function (PDF) of the property evaluated at 
the location of the measurement. This is exactly how we 
treated the likelihood factor in MJ11: the "datum" was 
the depth of the transit signal, and the PDF was deter- 
mined using the distribution of depths from the repre- 
sentative population simulations. 

The crucial step forward I take in this work is to take 
full advantage of fact that there is much more informa- 
tion in the shape of a transit signal than just its depth. In 
short, transiting planet and diluted eclipsing binary sig- 
nals, at a fixed period, generally do not look the same. 
This is the starting point and backbone of the Kepler 
team's BLENDER analysis, in which the goodnesses-of- 
fit of exhaustive simulations of putative false positive 
light curve models are compared to the goodness-of-fit 
of a transiting planet model; false positive models that 
do not produce acceptable fits are ruled out. 

In contrast to BLENDER, I do not fit physical models 
to light curves to rule out false positive scenarios. In- 
stead, I use the transit signal shape by extending the 
likelihood analysis of MJ11 into two additional dimen- 
sions: duration and slope of the signal, in addition to 
depth. I accomplish this by using our representative 
population simulations to define the three-dimensional 
probability distribution of these transit shape parame- 
ters, and then evaluating this PDF for the shape of the 
observed signal — the exact three-dimensional analogue of 
the likelihood calculation of MJ11. To enable this, I turn 
to the simplest of transit shape models: the trapezoid. 

For each instance in the representative population of 
each scenario, I follow a two-step procedure. First, I 
model the exact shape of the eclipse (accounting for every 
detail of the eclipsing system including the eccentricity 
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and orbital orientation provided by the simulation) us- 
ing a limb-darkened eclipse model ( Mandel fc Agol|2002[ ) 
with appropriate quadratic coefficients ( (Jlaret| |2000 ) . 
taking into account the 30 minute Kepler iong-cadence 
integration time. Secondly, I fit that physically mod- 
eled light curve with a simple trapezoid function. I pa- 
rameterize the trapezoidal shape with three parameters: 
depth, total duration (from first to last contact), and 
the ratio of total duration to ingress time, or "slope," 
which parametrizes the shape of the signal: i.e. whether 
it is "box-shaped," "V-shaped," or somewhere in be- 
tween. From each population I thus obtain a three- 
dimensional scatter plot of shape parameters, which I 
turn into a three-dimensional PDF using Gaussian ker- 
nel density estimation (using the 'gaussJcde' routine from 
the scipy.stats Python librarjj^]). 

Once this PDF has been created for each scenario, 
the each likelihood may be calculated as well. To this 
end, I fit the trapezoidal shape to the observed phase- 
folded transit light curve, using a Markov C hain Monte 
Carlo routine ( |Foreman-Mackey et al.||2012 ) in order to 
obtain reliable error bars (important for signals with 
lower signal-to-noise ratios). From the MCMC chains 
I construct a three-dimensional posterior PDF, and then 
calculate the likelihood by multiplying the two PDFs 
("model" and "data") together and integrating over all 
three dimensions. This is just like the simple likelihood 
calculation discussed above, except that instead of eval- 
uating the model PDF at a single point, I integrate over 
the uncertainty in the data values. The procedure is il- 
lustrated in the figures in £|4j which also give a sense of 
what the 3D shape distributions look like for different 
scenarios. 

Note that nowhere in this process do I attempt to de- 
rive any physical properties (e.g. impact parameter, a/i?* 
etc.) from the trapezoidal fit, nor do I at any point fit a 
physical transit model to the observed data — the trape- 
zoid function is purely descriptive, and precise physical 
properties of the transiting planet are unnecessary in or- 
der to calculate its FPP. I also do not use the candidate's 
period in the likelihood analysis, apart from the fact 
that all the simulated scenarios are simulated to have the 
same period as the candidate. In other words, I do not 
attempt to compare the period distributions of the vari- 
ous false positive scenarios with an assumption regarding 
the planet period distribution. The only question I aim 
to answer with this likelihood calculation is "How much 
does this signal look like each scenario?" and ultimately 
"Does this signal look much more like a transiting planet 
than a false positive?" 

3.4. Final Calculation 

Besides the priors and likelihoods for all the false pos- 
itive scenarios, calculating the FPP for a transit signal 
requires assumptions about the true distribution of tran- 
siting planets. Here my strategy departs from MJ11. In 
MJ11, we calculated FPPs under the assumption of an 
overall 20% planet occurrence rate, and a distribution of 
planet radii dN/dR ~ R~ 2 . In order to avoid such blan- 
ket assumptions of planet occurrence rates and radius 
distributions in this work, I proceed as follows. 

5 http:/ /docs. scipy.org/doc/scipy/ 
reference/generated/scipy.stats. gaussian_kde.html 



Rather than use a generic distribution of planet radii in 
order to simulate the representative population of tran- 
siting planets that I need (in order to determine the TP 
prior and likelihood necessary to complete the FPP cal- 
culation), I instead simulate a distinct "radius bin" pop- 
ulation, customized to the signal in question. In other 
words, I draw planet radii from a uniform distribution 
between 2/3 and 4/3 the best-fitting planet radius and 
simulate the representative population of transit signals 
using these radii. Of course, this nearly always makes 
the depth of the signal lie very near the middle of the 
depth distribution; this is by design. If the duration 
and slope parameters also lie near the middle of the TP 
distribution while being on the outskirts of all the FP 
distributions, this may by itself be evidence enough to 
validate the planet, as I demonstrate in <Q 

While simulating the planet population to have just 
the right size to match the observed signal may seem to 
give an improper advantage to the planet model, this is 
perfectly reasonable as long as the prior is treated appro- 
priately. No longer may a generic planet occurrence rate 
(e.g. "40% of stars have planets") be used; instead, the 
relevant occurrence rate is the assumed fraction of stars 
that have planets in the radius bin in question. I call this 
the specific occurrence rate f p and is defined as an 
integral over the planet radius distribution function $#: 



fp(Rp) 



1.3R„ 



$ R (R)dR 



(4) 



0.7R„ 



What sort of assumption is reasonable for this f p l For 
example, if $p(R) oc R~ 2 from 0.5 and 20 Earth radii, 
34% of the probability lies between 0.7 and 1.3 i?®, 
whereas only 3.4% of the probability lies between 7 and 
13 i?®. Thus if an overall 40% occurrence rate were 
assumed with this radius distribution, the specific occur- 
rence rate of a ~1 i?® planet should be ~14% and the 
specific occurrence rate of a 10 i?® planet should be 
1.4% (see Figure [2] for more illustration). 

However, the motivation for this "radius bin" strategy 
is to avoid relying on an assumption of an overall planet 
occurrence rate and radius distribution, and integrating 
a generic power law to obtain a specific occurrence rate 
does not accomplish this. Instead, consider the following 
rearrangement and simplification of Equation [2] 



FPP 



L 



FP 



L 



FP 



fpL 



p^TP 



where 



L 



FP 



(5) 



(6) 



with i representing all of the false positive scenarios, 
f p being the true specific occurrence rate, and Ltp 
equalling tttpCtp if the specific occurrence were unity 
(i.e. assuming every star has a planet in this radius bin). 
Equation [5] may be further simplified as 



FPP = 



1 + fpP 



(7) 



where P = Ltp / Lpp. This allows me to parametrize the 
FPP in terms of the unknown specific occurrence rate f p 
and the factor P, which is independent of assumptions 
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Number of planets per star = 0.20 
Number of planets per star = 0.40 
Number of planets per star = 1.0 




0.00 



Fig. 2. — The specific occurrence rate f p as a function of radius, 
for three examples of generic radius distributions, all scaling as the 
inverse square of the planet radius. I define the specific occurrence 
rate at a particular radius R p to be the integral of the planet radius 
probability density function between 0.7R P and 1.3i? p (Equation 
Hi. Even though the use of the specific occurrence rate in the anal- 
ysis presented in this paper makes it independent of assumptions 
of the overall planet rate or the exact shape of the radius distri- 
bution, this illustrates what reasonable values of /„ might be. For 
example, it is clear that f p = 0.01 is quite conservative for nearly 
any planet size, especially for small planets. 



about either planet occurrence or the true planet radius 
distribution. 

The final result of this validation method may thus 
be presented as simply this factor P, which contains all 
the information necessary to calculate the FPP based on 
an assumed specific occurrence rate (Equation ff\. This 
lends ease to interpretation, as for large values of P, the 
FPP is very nearly l/(f p P). Thus if P = 1000, then 
under an assumption of f p = 0.5, the FPP is 1 in 500. 
For f p = 0.2, FPP would be 1 in 200, and so on. 

Another useful presentation answers the following 
question: at what value of the assumed specific occur- 
rence rate would the FPP be low enough to consider the 
planet validated? This is simply a matter of choosing a 
target FPP (e.g. FPPy for "validation FPP") and solv- 
ing for f p in Equation [7j 



fp,V 



1-FPP 



v 



P ■ FPP, 



(8) 



which may be approximated by f p y = [P ■ FPPy) . 
For example, for a target FPP of 0.5%, P = 1000 re- 
quires f Pt v = 0.2 to validate, while P = 2x 10 4 requires 
only f„ y = 0.01. Re cent results f rom both RV surveys 



( |Howard et aL||2010 ) and Kepler ( |Howard et al.||2011 1 
suggest that over 40% of stars have planets of some sort 
within ^50 day orbits, and that the frequency of planets 
rises with decreasing radius. Other work indicates both 
that the average n umber of planets per star may even be 
> 1 ( Youdin|2011 ) and that f raction of stars with planets 
may also be close to unity (|Cassan et al.| |20"l2"| ). Thus, 
if fp,v for a particular transit signal is measured to be 
significantly below the curves plotted in Figure [2] (e.g. if 
it is 2-3% for a signal of a 2R (S transit), the planet may 
be considered securely validated. 
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Fig. 3. — The phase-folded Kepler photometry of Kepler-19b 
(KOI 84.01). The solid line illustrates the best- fitting trapezoid 
model used for the likelihood calculations (j ]3.3| . Note that the 
transit is clearly box-shaped, not V-shaped. 

4. TESTS 

In this section I present the results of applying this pro- 
cedure to a sample of Kepler Objects of Interest (KOIs), 
all of which have been investigated already by other 
means. I show in the following subsections that this 
analysis can both easily validate known planets (using 
no or very minimal follo w-up observations) and identify 
probable false positives. §4. 1| illustrates the procedure in 
detail using the example of Kepler- 19b, §4.2| describes the 
resul ts of analyzing 17 additional known Kepler planets, 
and §4. 3| discusses application to 13 known or suspected 
false positives. 

4.1. Kepler-1% 

Kepler-19b, or KOI 84.01, orbits a Sun-like star with 
a 3.5-day peri od and has a radius of 2.2 R®. The dis- 
covery paper (Ba llard et al.|[2011[ ) details the sequen- 
tial battery of tests used to rule out false positive ex- 
planations for its photometric signal: first the prelimi- 
nary tests possible with just the photometry (search for 
a secondary eclipse and measurement of the in-transit 
photocenter shift), and then the follow-up observations. 
The observation follow-up effort included reconnaissance 
spectroscopy at three epochs using the McDonald Ob- 
servatory 2.7m telescope, high-precision radial-velocity 
Keck/HIRES spectroscopy at 20 epochs (though only 8 
were used in analysis), adaptive optics (AO) imaging at 
the Palomar 200-in, speckle imaging at the WIYN tele- 
scope in two filters, and 16 hours of Warm Spitzer ob- 
servation to measure the color dependence of the transit. 
None of these follow-up measurements revealed any sign 
of a false positive, and BLENDER analysis combined all 
these constraints with light curve modeling and estimates 
of the planet and false positive priors (based on Kepler 
data itself) to calculate a final false positive probability 
of about 1 in 7000, validating the signal. 

For my analysis of Kepler-19b, I begin from the as- 
sumption that no secondary eclipse is detected in the 
photometry (at a level of 200ppm), and that photocen- 
ter analysis constrains any potential blend to be within 
2" at most. I then proceed through the steps detailed 
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Fig. 4. — The false positive landscape for Kepler-19b (KOI 84.01). Each of these plots illustrates a three-dimensional probability 
distribution for the trapezoidal shape parameters (depth 5, duration T, and "slope" T/t) for a false positive s cenar io (see ij2]for descriptions 
of the scenarios). Each of these distributions is made by simulating a statistically representative population ( j]3.1[ ) for a scenario and fitting 
the shape parameters to each simulated instance of the scenario. Each population begins with 20,000 simulated instances, and only instances 
that pass all available observational constraints are included in these distributions — in this case the constraints are that the blended star be 
at least 1 mag fainter than the primary target, the lack of an observed secondary eclipse deeper than 200ppm, and that the KIC g — r and 
2MASS J — K colors match within 0.1 mag. The pie chart for each scenario illustrates what fraction of the initial simulations pass these 
tests. On each plot the shape parameters of the transit signal are marked, with the 'X' showing the median of an MCMC fit, and error bars 
illustrating 95% confidence. The probability of each scenario is calculated by dividing the prior X likelihood for that particular scenario 
by the sum of prior X l ikelih ood for all the scenarios (including the planet scenario illustrated in Figure [5jl . Priors for each scenario arc 
calculated according to §3.2| and the likelihoods are calculated by integrating the illustrated distributions over the observed measurement. 



in fj3] use the target star's position and KIC photome- 
try to simulate the representative populations for each 
scenario considered, calculate the priors based on these 
simulations and various assumptions about stellar popu- 
lations, measure the trapezoidal shape parameters of the 
Kepler-19b signal itself, and calculate the likelihoods for 
each scenario based on these measured shape parameters 
and the distributions of shape parameters of the scenario. 
Figures [4] and [5] illustrate how the shape of the signal 
matches the typical shape of a transiting planet much 
better than any of the false positive scenarios. Assuming 
a 1% specific occurrence rate for planets like Kepler-19b, 
these calculations result in a FPP of 2.8%. Alternatively, 
a specific occurrence rate of about 6% would be required 
for FPP=0.5% — close to being securely validated. The 
most likely false positive is the HEB scenario (Figure 



When generating the representative population for the 
true transiting planet scenario for this signal, I use the 
stellar parameters for KOI 84 provided in Batalha et al.| 
(I2012[ ) (0.86 i? Q , 0.91 M Q ), with 20% uncertainties. 
These notably do not agree exactly with parameters pub- 
lished in the discovery paper, determined from spectro- 
scopic analysis (1.1 Rq, 0.97 M ). The effect of this dis- 
crepancy is visible in Figure [5] where the measured slope 
parameter (T/t) of the signal appears to fall slightly 
above the main distribution of the simulated population: 
a smaller star will result in a smaller T/t for a fixed value 
of R p /R+. The result is that the likelihood of the planet 
signal is calculated to be lower than what it would be if 
the stellar parameters of the simulated population were 
correct, leading to an overestimate of the FPP. Since 
accurate stellar parameters will not necessarily always 
be available, this circumstance makes for a realistic test 




log(J) r[days] 



Fig. 5. — The transiting planet scenario for Kepler-19b (KOI 
84.01). The plots illustrate the three-dimensional probability dis- 
tribution for the trapezoida l sha pe parameters for simulations of a 
representative population (j ]3.1| l of transiting planets around KOI 
84. The radius distribution ol planets in this population is chosen 
to be a narrow range centered on the radius derived for KOI 84.01 
assuming it is a bona fide transiting planet. The spread and shape 
of this distribution are caused by different-sized planets in the sim- 
ulation, different stellar properties (according to uncertainties in 
stellar radius and mass), and by variations in orbital inclination, 
which changes duration and slope in a correlated manner (bottom- 
right panel) . The assumed occurren ce ra te of planets in this radius 
bin for this calculation is 1% (see j ]3.4| for more discussion about 
this "specific occurrence rate"). The probability of the Kepler-19b 
signal being a transiting planet, given the false positive landscape 
illustrated in Figure H] (constrained by only the signal photometry 
and KIC colors) is about 97%, giving FPP = 0.03. The signal 
does not fall quite exactly in the middle of the T/t distribution 
because this population simulation uses photometrically estimated 
physical properties (mass, radius) for the the host star, rather than 
the more accurate spectroscopically derived properties. Note that 
uncertainties like this will tend to decrease the planet likelihood, 
and thus will contribute to overestimating the FPP. 

case. 

The next step in my re-analysis of the Kepler-19b FPP 
is to pretend that a single follow-up observation is avail- 
able: a high-resolution spectrum from which T e g- and 
log g of the host star can be accurately measured, to a 
precision of 80 K and 0.1 dex, respectively. The effect 
of this constraint is to rule out those binary and hier- 
archical scenarios from the population simulations that 
were initially allowed (since the overall colors matched 
the KIC colors), but whose temperatures or surface grav- 
ities are >3cr discrepant from this measurement. Figure 
[6] shows how the distribution of shape parameters for the 
HEB scenario changes under this constraint such that the 
measured Kepler- 19b parameters are even more on the 
outskirts — the HEB instances that could mimic a tran- 
sit shape were exactly the ones ruled out by the stellar 
characterization. Using the likelihoods and priors up- 
dated for this constraint, the specific occurrence rate as- 
sumption necessary to result in an FPP of 0.5% (/ P ,y) 
for Kepler-19b is only 1.8% — small enough to result in a 
secure validation. 

To investigate whether a single AO observation would 
have a similar effect, I remove the spectroscopic con- 
straint and submit the simulated populations to the con- 
straint of a generic deep AO image (AK = 8 at 077), 
ruling out all instances the false positive scenarios that 
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Fig. 6. — The hierarchical triple eclipsing binary (HEB) scenario 
for Kepler-19b (KOI 84.01) under the constraint of spectroscopic 
characterization of the host star. The plots illustrate the three- 
dimensional probability distribution for the trapezoidal shape pa- 
rameters for simulations of a representative population (3.1) of 
HEB false positive scenarios, but only including those instances 
in which the spectroscopic properties (T e ff, logo) of the primary 
star match those that were measured for KOI 84. Compare this 
distribution to Figure |4(c)| the systems whose shape parameters 
overlapped with the properties of the measured signal are not al- 
lowed by the spectroscopic characterization — these correspond to 
the higher-mass systems illustrated in FigureJT] This constraint 
allows for the signal to be validated, as the HEB scenario was the 
most-likely FP scenario before including the follow-up spectral in- 
formation. 

would be identified by this observation. This AO con- 
straint alone also decreases the FPP enough to claim 
validation, though not quite as strongly as the specto- 
scopic constraint: f p y — 0.03. The combination of the 
two constraints results in f p y = 0.1%. 

The conclusion of this investigation is striking: secure 
validation of the Kepler-19b signal with these methods 
would have been possible using either of two single-epoch 
follow-up observations: a high-resolution spectrum to 
measure the spectroscopic parameters or an AO image 
to constrain blends. With both of these observations, 
validation would be secure beyond doubt. 

4.2. Kepler planet test sample 

While Kepler-19b is an eye-opening individual example 
of the potential of this analysis to fully validate transit- 
ing planet candidates with very modest follow-up invest- 
ment, it is just a single example. I thus identify a broader 
sample to test in a similar manner: 17 other known Ke- 
pler planets, selected to be all of the officially designated 
Kepler planets that do not show significant transit timing 
variations. 9 of these were confirmed with RV measure- 
ments, and the additional 8 were BLENDER- validated. 

For each of these transit signals I follow the exact same 
analysis sequence as described above for Kepler-19b, us- 
ing the same generic assumptions of a 200ppm secondary 
eclipse limit and a blend radius of 2" . I find that 4 of the 
17 would have been validated with no follow-up obser- 
vations at all, 6 require only stellar characterization to 
validate, 9 require only AO imaging, and all except 7 (all 
giant planets but 1) can be validated with the combina- 
tion of both constraints. These results are summarized 
in Table [TJ which presents the values of f p y for each sig- 
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Test sample: 



TABLE 1 

Non-TTV Confirmed Kepler planets 1 



Name 


Method 


fp,V 


— PjV — 


f AO 
J p y 


jspec, AO 
Pj_ 


Kepler-4b 


RV 


0.006 


0.013 


0.003 


0.010 


Kepler-6b 


RV 


0.161 


0.163 


0.162 


0.163 


Kepler- 7b 


RV 


0.068 


0.140 


0.052 


0.112 


Kepler-8b 


RV 


1.227 


1.087 


0.808 


0.781 


Kepler-lOb 


RV/BLENDER 


0.014 


0.014 


0.000 


0.000 


Kepler-lOc 


BLENDER 


0.014 


0.012 


0.002 


0.000 


Kcpler-llg 


BLENDER 


0.289 


0.200 


0.140 


0.061 


rvcplcr-lzD 


rlv 


n nee 
U.UoO 


U.Ulo 


U.UoO 


U.Ulo 


Kepler- 14b 


RV 


0.010 


0.000 


0.000 


0.000 


Kcpler-15b 


RV 


0.293 


0.116 


0.193 


0.104 


Kepler- 17b 


RV 


0.109 


0.131 


0.094 


0.118 


Kepler-18b 


BLENDER 


0.460 


0.241 


0.189 


0.002 


Kcpler-19b 


BLENDER 


0.058 


0.018 


0.030 


0.001 


Kcpler-20e 


BLENDER 


0.071 


0.056 


0.014 


0.001 


Kepler-20f 


BLENDER 


0.178 


0.176 


0.009 


0.007 


Kcpler-21b 


BLENDER 


0.010 


0.010 


0.001 


0.001 


Kepler-22b 


BLENDER 


0.048 


0.051 


0.004 


0.001 


TOTALS 


17 (all) 


4 


6 


9 


10 



1 The results of applying this validation procedure to a sample of 
known Kepler planets, 9 of which were confirmed with RV measure- 
ments and 9 by BLENDER analysis (one by both). f p y is the spe- 
cific planet occurrence rate required to result in an FPP of 0.5% 
(Equation [sj . When f p ,v is significantly below a reasonable oc- 
currence rate estimate, (Figure |2j, I consider the planet validated 
(marked in bold). The 'spec' and 'AO' superscripts indicate the 
effect of incorporating spectroscopic and imaging follow-up observa- 
tions. Most of the planets are validated with both types of follow-up; 
the ones that are not are nearly all giant planets. 
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Fig. 7. — The phase-folded Kepler photometry of KOI 552.01. 
The solid line illustrates the bes t- fitting trapezoid mod el used for 
the likelihood calculations (j ]3.3| l. |Bouchy et al.| | |2011[ l measured 
large RV variations of the host star suggesting that triis signal is 
caused by an eclipsing M-dwarf rather than a planet. Note that 
the signal is clearly more V-shaped than box-shaped. 



nal under the various constraints. A signal is considered 
validated when f p y is significantly below the expected 
planet occurrence rate for a planet of that size — the val- 
ues low enough for validation are bolded in the table. 
These results are quite encouraging, suggesting that per- 
haps many future Kepler planet validations will require 
only minimal follow-up observations. 



Despite the encouragement of the above results, it is 
fair to ask whether this analysis is skewed such that it al- 
ways returns a low false positive probability, and thus au- 
tomatically tuned for easy planet validation. To address 
this concern, I investigate a sample of Kepler candidates 
known to be false positives. 

KOI 552.01 (Figure is a Jupiter-sized candidate 
whose host star has a KlC-estimated T e & — 6018 K. 
This was selected as one of the candidates for follow- 



up wi th the SOPHIE spectrograph, and |Bouchy et al.| 
(2011 1 measured it to have ~km/s RV variation between 



two epochs. This suggests that the signal is likely caused 
by an eclipsing M-dwarf star rather than a giant planet. 
In addition, analysis of the spectra suggests that the tem- 
perature of the star is 6500 K, significantly hotter than 
the KIC estimate. 

For KOI 552.01 1 calculate FPP=0.90 (assuming a spe- 
cific occurrenceofl%, appropriate for Jupiter-sized plan- 
ets ( Wright et al.|2012 |) — in other words, this procedure 
clearly identifies this signal as a very likely false positive 
using only the photometry, and no follow-up information 
at all. Figure[H]illustrates how well the shape of the signal 
is described by a hierarchical system with an eclipsing bi- 
nary. Conveniently, an HEB explanation for KOI 552.01 
explains not only the eclipse signal but also the discrep- 
ancy between the KIC and spectroscopic T e s estimates: 
if this system is indeed a triple system with a primary 
with T e ff = 6500 K and two cooler stars, then trying to 
interpret the broadband photometry of the whole sys- 
tem as a single star would naturally result in a cooler 
temperature estimate. 

Similar analysis of 12 other KOIs that have been iden- 
tified as false positives gives similar results, with all but 
one having FPP > 20%, and six having FPP > 80% 
(T able |2l Of these , 11 were identified as false positives 



by Sanlerne et al. J 2012) and one (1187.01) was iden- 



tified by|Colon et~aL"l2"0l2) to have a color-dependent 



4.3. Known false positives 



transit depth. 1 assume a specific occurrence rate of 1% 
for all these candidates. All but 1187.01 have transit 
depth > 0.4%; this is due to the way that the SOPHIE 
follow-up sample was chosen. Notably, many of these 
signals have T/r parameter very close to 2, indicative 
of a V-shaped transit shape. This demonstrates that in- 
corporating transit shape into this analysis allows this 
procedure not only to validate planets, but also to reli- 
ably identify likely false positives. 

4.4. Testing assumptions and simulation variance 

As mentioned in |3.1| the false positive calculations 
that this procedure delivers are dependent on a set of as- 
trophysical assumptions. In addition, apart from a pri- 
ori assumptions, the calculations are subject to intrinsic 
variance due to the fact that FP distributions used to 
calculated the likelihoods are the result of Monte Carlo 
simulations. To test the degree to which qualitative re- 
sults depend on these effects, we repeat the calculations 
for the known planet sample many times, using differ- 
ent sets of assumptions. I repeat each test three times, 
with the full range of results for the final f p y numbers 
illustrated for each test in Figure [9] The tests are the 
following: 

• The original simulations under the original assump- 
tions (repeated three times). 
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Fig. 8. — The hierarchical triple eclipsing binar y system scena rio 
for KOI 552.01. This KOI was measured by |Bouchy et al.| l |201l) to 
have ~km/s radial velocity variation between two epochs, suggest- 
ing a stellar eclipse rather than a transiting planet — an observed 
astrophysical false positive. The analysis presented in this paper 
gives an FPP of >90% for this signal, and this figure illustrates 
why: the shape of the signal is exactly consistent with what would 
be expected from a hierarchical triple system including an eclipsing 
binary. 



TABLE 2 

Test sample: False positives 1 



KOI 


FP identification method 


<5(%) 2 


T/r 3 


FPP 4 


552.01 


RV variation 81 


0.8 


2.6 


0.95 


1187.01 


color-dependent transit 10 


0.2 


2.4 


0.82 


190.01 


Blended CCF; RVs c 


1.1 


3.4 


0.22 


340.01 


RVs c 


2.1 


5.5 


0.30 


418.01 


Blended CCF; RVs c 


1.2 


3.4 


0.85 


419.01 


RVs; sec. eclipse only c 


0.8 


2.4 


0.96 


425.01 


Blended CCF; RVs c 


1.2 


2.8 


0.61 


607.01 


RVs c 


0.7 


3.7 


0.32 


609.01 


double-lined binary 


0.4 


2.5 


0.96 


667.01 


blend (DSS photometry) 
RVs; sec. eclipse only c 


0.9 


3.8 


0.07 


698.01 


0.7 


2.5 


0.96 


1786.01 


RVs c 


0.8 


5.8 


0.48 



1 The results of applying this analysis on a sample of KOIs that have 
been identified as false positives with various follow-up observations. 
Nearly all are identified as likely false positives. 

2 Transit depth 

3 The ratio of the total transit duration to the ingress/egress duration. 
A value close to 2 is a V-shaped signal; a larger value indicates a more 
box-shaped transit. 

4 False positive proba bility, assuming a specific occurrence rate of 1%. 
a lBouchy et~aL"n2011 || 
b Colon et al.|(|20 12t 
c Santcrne et al. 



The original simulations under the original assump- 
tions, but using N — 100, 000 for the simulations 
(rather than N = 20, 000) 

N = 100, 000, changing the mass ratio distribution 
to be flat between 0.2 and 1 (rather than between 
0.1 and 1) 

N = 100, 000, using the Dartmouth stellar models 
(Potter et al.|200"8) in place of the Padova models. 



• N — 100,000, increasing the "color tolerance" of 
the EB and HEB scenarios to 0.2; i.e. allowing all 



Fig. 9. — The results of testing the effects of sample variance and 
changing various assumptions on the false positive calculations de- 
scribed in this paper. Each colored vertical line represents the full 
range of f Pl v (t ne specific occurrence rate assumption required for 
validation) over three iterations of the procedure for each planet 
from Table [l] ordered by increasing radius. The five different ver- 
tical for eacn planet represent 5 different experiments, varying the 
size of the Monte Carlo simulations and different input astrophysi- 
cal assumptions. All of the planets that were claimed as validated 
by the original analysis (the final column of Table [TJ remain vali- 
dated for each iteration of each of these tests. 

FP scenarios that match within 0.2 mag of the g — r 
and J — K colors, rather than 0.1 mag. 

As seen in the figure, in only two cases do the range 
of results cross the nominal "validation threshold" : 
Kepler-llg occasionally drops into validation territory, as 
does Kepler- 7b, for one instance of the N = 20, 000 sim- 
ulation. None of the planets validated with both follow- 
up observations from Table [T] cross over into the non- 
validation regime. We conclude that larger simulations 
are always better if possible, to reduce variance, but that 
N = 20, 000 is usually sufficient. While the exact value of 
FPP may change by a factor of a few from repetition to 
repetition, secure validations remain secure validations. 

There are other input assumptions not tested by this 
exercise, but these are mostly trivial assumptions that 
can be tested on a case-by-case basis simply by adjust- 
ing the multiplicative factors that go into the prior cal- 
culation for a given scenario. These include stellar bi- 
nary and triple fractions, the fraction of binaries that are 
in "short" orbits, the occurrence rate of planets around 
background stars, and the density of background stars 
returned by TPJLEGAL. Adjusting any of these num- 
bers changes the prior factor for that scenario, and cor- 
respondingly changes the false positive probability. If the 
FPP is small and one false positive scenario dominates, 
the overall FPP is directly proportional to the prior fac- 
tor for that scenario (see the discussion in §5 of MJ11); 
e.g., if the HEB scenario is the most likely false positive 
for a particular candidate, doubling the assumed stellar 
triple fraction will approximately double the FPP (or, 
equivalently, double the f p y required for validation). 

5. RELATION TO MORTON & JOHNSON (2011) 

While I have demonstrated that the results from this 
newly updated FPP analysis are reliable both with 
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known planets and known false positives, I now briefly 
discuss why these r esults may differ from the individual 
FPPs published in Morton & Johnson (2011). First of 
all, it is important to understand that the goal of the 
previous work was to demonstrate that, in general, Ke- 
pler candidates that pass all possible initial photometric 
vetting are very likely to be planets. That is, the MJ11 
analysis is only strictly valid for signals that are not ob- 
viously V-shaped and whose light curves have been de- 
clared free from secondary eclipses. 

As i t turn s out, many of the candidates from [Borucki | 
et al. (2011 ) were not in fact fully vetted. This was made 
cle ar in that paper, but this fact was not reflected in 
the |Morton fc Johnson] ( |2011[ ) KOI FPP table. In other 
words, there are some signals that are clearly V-shaped (a 
number of these are n ow in the known false positive sam- 
ple discussed in ^4.3), and likely also signals that show 



faintly detectable secondary eclipses. For such KOIs, it 
is probable that the final FPP will turn out to be sig- 
nificantly higher, as is the case for t he sa mple of false 
positives detected by Sante rne et al. ( 2012[) . 
The analysis of Morton & Johnson ( 2011 ) was neither 



a planet validation procedure nor a false positive detec 
tion procedure. Nor was its purpose to claim that the 
Kepler results are completely immune from the problem 
of false positives. Instead, its goal was to demonstrate 
that the a priori FPP for fully vetted Kepler candidates 
is typically low, thus suggesting that the overall sample 
of candidates generally reflects the properties of the true 
sample of planets (barring the existence of a pathologi- 
cally large number of V-shaped signals among the candi- 
date list). On the other hand, this work, hav ing advanced 
the techniques of 
to validate candic 
an individual basis. 



Morton & Johnson (20111, is intended 
ates, using all available information on 



6. CONCLUSIONS 

Transiting exoplanet surveys continue to grow much 
faster than the capabilities of traditional follow-up infras- 
tructure. While the observations required to positively 
confirm candidates have always been the bottleneck for 
transit surveys, the Kepler mission has made this issue 
much more acute. Future ground- and space-based sur- 
veys will doubtless continue to produce orders of magni- 
tude more candidates than can be securely confirmed ei- 
ther with radial velocity or transit timing variation mea- 
surements. As a result, the onus of confirmation for the 
majority of transiting planet candidates has shifted ir- 
revocably from dynamical confirmation to probabilistic 
validation. 

With probabilistic validation playing such a central 
role, the procedure I describe in this paper enables a 
revolution in how transit surveys could operate: validat- 
ing more planets, using fewer follow-up resources. Recall 
what I have demonstrated about this procedure: 



• It is efficient: an end-to-end FPP calculation for a 
transit signal takes only about 10 minutes as cur- 
rently implemented — and there is room for further 
computational optimization. 

• It often requires only one or two single-epoch 
follow-up observations (either spectral characteri- 
zation, AO imaging, or both — and sometimes nei- 
ther) to validate a true transiting planet. 

• It can reliably identify false positives. 

The benefits of applying this analysis to a large sample of 
candidates from a transit survey such as Kepler are evi- 
dent. Spectroscopic follow-up can be prioritized in order 
of increasing FPP, likely leading directly to many vali- 
dations. Preference for AO imaging can then be given 
to those candidates not validated with a spectrum. RV 
observations of likely false positives can be avoided, and 
more time devoted to RV measurements of likely planets 
or characterization spectroscopy. And the most resource- 
intensive follow-up tools, such as multi-band photome- 
try, validation-inspired RV measurements, and the full 
BLENDER analysis, can be reserved for the most stub- 
born candidates — those that are not likely false positives, 
but resist validation even after spectral characterization 
and AO imaging. A good example of a candidate ideal 
for this last-stage analysis would be one with a residual 
FPP of 5-10% or so, for which AO imaging reveals a faint 
blended companion. 

With regard to Kepler, I intend to apply this proce- 
dure to all of the remaining unvalidated KOIs, using 
whatever follow-up observations are available. There are 
several ongoing projects to spectroscopically characterize 
and obtain AO images of large numbers of KOIs; doubt- 
less these efforts have already provided the measurements 
necessary to validate hundreds of planets. 

The ultimate goal of this project is to make self- 
contained, stand-alone transiting planet validation soft- 
ware available to the community. This will allow present 
and future transit surveys to apply this analysis at an 
early stage, hopefully enabling follow-up resources to be 
optimized and planet validations to be expedited. The 
hope is that with fewer observational resources dedicated 
to the task of planet validation, more can be devoted to 
detailed characterization of individual systems of par- 
ticular interest. In addition, with large numbers of se- 
curely validated transiting planets, statistical population 
studies will become possible without worrying about any 
residual effects from false positive contamination. 

TDM acknowledges helpful discussions with John 
Johnson, Erik Petigura, Jason Eastman, Justin Crepp, 
Jon Swift, John Carpenter, Steve Bryson, and many Ke- 
pler team members throughout the course of this re- 
search. He also thanks the referee Scott Gaudi, who 
waived anonymity, and whose suggestions led to substan- 
tial improvements to this manuscript. 
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